web stats
 
   
 
Selected Research Project
 
Project Title Building a New Methodology of Assessment of Differential Item Functioning
 
Principal Investigator

Professor Wang Wen Chung
 
Area of Research Project Educational Measurement and Assessment

 
Project Period
From 11/2010 To 11/2012
Objectives
  1. Critique existing differential item function (DIF) assessment methods in the establishment of a common metric by providing didactic examples to demonstrate their mechanisms and assumptions, and explicate why and when such methods produce misleading results;
  2. Develop new methods for selection of DIF-free items and evaluate their performances;
  3. Propose the DIF-free-then-DIF (DFTD) strategy, implement this strategy using various DIF assessment methods, and demonstrate its advantages over alternative methods;
  4. Exemplify implications and applications of the new strategy through systematic and secondary real data analyses; and
  5. Build a new methodology and begin a new era of DIF assessment.
Methods Used
  • Theoretical analysis of existing methods;
  • Invention and evaluation of methods for selection of DIF-free items;
  • Implementation of the DFTF strategy in IRT-based methods and other traditional methods; and
  • Applications of the DFTD strategy.
Summary of Findings

We have developed a new strategy for the detection of DIF. The new strategy was found to be more effective than traditional approaches, especially when tests consisted of a high percentage of DIF items. Some of the research results have been presented in academic journals and conferences, and several manuscripts are under preparation.
Impact
  • Clarification of mechanisms and assumptions of exiting DIF assessment methods would allow users to recognize their fallacies and limitations easily.
  • The innovative strategy would enable more accurate DIF assessment compared to existing methods, thereby ensuring better test fairness and measurement quality.
Selected Publications Related to the Study
  • Wang, W.-C., Shih, C.-L., & Sun, G.-W. (in press). The DIF-free-then-DIF strategy for the assessment of differential item functioning. Educational and Psychological Measurement.
  • Tsai, C.-C., Wang, W.-C., & Shih, C.-L. (2011, July). The performance of DIF-free-then-DIF strategy in MIMIC method. Paper presented at the International Meeting of the Psychometric Society, Hong Kong.
  • Chang, H.-C., Wang, W.-C., & Shih, C.-L. (2011, July). How many anchor items should be selected in DIF-free-then-DIF strategy? Paper presented at the International Meeting of the Psychometric Society, Hong Kong.
Biography of Principal Investigator
Professor Wang Wen Chung is the Director of the Assessment Research Centre and Chair Professor of Educational and Psychological Measurement. Prior to joining HKIEd in 2008, he was a Distinguished Professor and Head of the Department of Psychology at National Chung Cheng University in Taiwan. He obtained his Ph.D. from the University of California, Berkeley. He was a Fulbright grantee in 2005, and a visiting professor at UC Berkeley in 2005 and 2007. He has received Research Excellence Awards from the National Science Council and National Chung Cheng University, as well as the Mu-Dou Award (¤ìÅM¼ú) for his contribution to education. He has published more than 140 refereed journal articles, in addition to several books and chapters. He is an editorial board member of several journals and a reviewer of many journals. Since joining HKIED in 2008, he has received (as principal investigator) three research grants from the General Research Fund and one research grant from the Public Policy Research Fund.
Funding Source

General Research Fund